Infinite Horizon Discounted Cost Problems

• We often approximate a large number of periods, even if the horizon is known and finite, by assuming an infinite number of periods, and hope that this assumption will simplify the solution. Indeed, even if the general theory becomes more involved, the solution obtained often is simpler and has important computational and conceptual advantages: in particular, the optimal policy is often stationary (the same function at every stage). For example, recall our discussion at the end of chapter 4 on linear quadratic Gaussian problems, where we mentioned that the optimal infinite horizon controller gain is constant and requires solving only one algebraic Riccati equation, whereas for the optimum finite horizon problem one must compute and/or store a different gain matrix for each time step. If the number of stages is large, we saw by an example that the difference in performance is typically negligible.